About 99 million years ago, a strange insect was getting ready for its next meal.

But before it could eat, the dripping resin of a tree killed both predator and prey, sealing them off from the world.

That predator was an ancient species of ant called Ceratomyrmex ellenbergeri, and it had the same six legs and segmented body that we’d recognize from an ant today.

But there were two major differences in its appearance: its huge, scythe-like jaw and the horn coming out of its head.

And pinned between these two structures was its prey - an ancient ancestor of cockroaches.

Now, this bizarre predator belonged to a group of ants commonly known as “hell ants.” And although they’re extinct, back in the Cretaceous Period there were as many as seventeen species of them.

They lived all around the world, and they managed to survive for at least 20 million years, so they were doing pretty well for a while — maybe thanks to their unique hunting skills.

But they’re gone now, and we’re still trying to figure out why.

Today, modern ants live in huge colonies, have distinct roles within their nests, and help care for young even if the larvae aren’t their direct offspring.

Did the hell ants fit this mold, or were they something else entirely?

The answer could help us understand why the social ants we know today are so successful and maybe why the hell ants ultimately weren’t -- it might come down to the evolution of teamwork.

The hell ants are some of the earliest ants known in the fossil record, and the first one was found in an amber mine from northern Myanmar about 100 years ago.

But that first specimen wasn’t actually identified until 1996, so almost everything we know about these insects comes from the last 25 years.

Each species had its own adaptations, whether it was enormous mandibles and curving headgear that helped them capture prey, or sharp horns for stabbing it.

But all of the species of hell ants shared one thing in common: jaws that moved vertically rather than horizontally.

And this is strange, because every species of ant known today has mandibles that move horizontally—from side to side.

We might think of modern ant mandibles as more like pincers than jaws, but hell ants seemed to use their horns like an extension of their mouths.

The jaws moved upward, and the horns acted as a stable structure to pin things against.

And hell ants are actually one of only two known insects with jaws that work vertically along with some kind of headgear, making it a pretty unique adaptation.

But as more specimens were discovered, experts also realized that there seemed to be different castes of the same species of hell ant—which is an important trait in living ants.

Castes are a strange feature of ants, in which members of the same species have different body types.

For example, there might be a queen with a large body and wings, workers with smaller bodies and no wings, and soldiers with bigger jaws, all within the same species.

Some of the castes are even born sterile, without the ability to reproduce—they just work to take care of the colony and the offspring of other ants.

And ants today are well-known for being eusocial insects—they have a division of labor, create colonies with multiple generations of adults, and many of the adults care for young that aren’t their own.

Now, while entomologists can understand modern eusocial ants by studying colonies in action, it gets a bit trickier with extinct groups, like the hell ants, because we can’t directly observe their behavior.

So, how do you tell if a hell ant was part of this wonderfully complex eusocial structure?

Well, scientists have found a number of these ants trapped in amber that can give us some clues about their social behavior.

Specimens from France show one ant that had wings, while others had no wings, but otherwise they were almost identical.

And a specimen from Myanmar preserved little torn wing nubs that may indicate it had recently shed it’s wings — which happens in some modern ants after a queen goes on a mating flight and settles down to start a new colony.

Scientists think these specimens might be evidence of castes and a reproductive division of labor, since modern ants with different body shapes do different types of work.

But!

It’s really hard to be sure from the fossil record whether the different body types all belong to the same species, or if they come from different species.

Another piece of evidence scientists look at is the presence of a metapleural gland that most ants have today.

This gland secretes fluids that are thought to help the ants keep clean, as well as communicate with each other, and this communication is thought to be another part of eusocial behavior.

Now, if ancient ants had this same type of gland, then maybe they were also communicating.

And guess what?

All Cretaceous ants, including hell ants, had a metapleural gland.

So, the presence of this gland and potential castes both suggest that they practiced some form of eusociality.

But things get complicated when it comes to the bigger picture of ants in the Cretaceous.

For example, take several pieces of Burmese amber that were all found close together and analyzed in 2016.

In one of the pieces, there are twelve worker ants all trapped together, eleven of one species and one hell ant.

In another piece, there are 21 ants from three different species.

And that’s kind of a big deal, because ants were much rarer during the Cretaceous than they are today.

They never make up more than 3% of the insects in deposits where they’re found.

So it would’ve been extremely unlikely for multiple ants of the same species to be trapped in the same piece of amber unless those ants were interacting with the other members of their species in some way.

But hell ants often appear trapped in amber alone.

So, does that mean hell ants were less social than other Cretaceous species?

It’s hard to say.

While hell ants most likely practiced some form of eusociality, they might not have had all the elements of highly eusocial insects.

Based on their appearance in the fossil record, some scientists have proposed that hell ants might’ve lived in small colonies with fewer than 100 individuals, and the castes might not have been quite as distinct as in modern ants.

Either way, hell ants are extinct today, and modern eusocial ants are everywhere.

So was true eusociality the thing that made the difference?

Because, it seems like a weird strategy from the perspective of natural selection - why would sterile individuals evolve in the first place?

Well, here we turn to what’s known as Hamilton’s rule of kin selection.

Hamilton’s rule boils down to this: Say a colony of related ants benefits from the ants helping each other.

While this help might not impact an individual ant’s offspring, the overall benefit to the colony will be higher than the cost to that one individual ant.

So, we end up with sterile individuals who work to ensure the survival of the group.

And there are definite advantages to joining a group, like more food, increased nest defense against predators or parasites, and the potential to inherit established nest sites.

But others have argued against kin selection by saying that high relatedness is more likely to be a consequence of eusociality, not a reason it evolves.

In other words, ants don’t help each other’s offspring because they’re the “aunts” or “uncles” of those offspring—they were already helping each other because of the benefits to the entire colony, and the relatedness just happens to be one byproduct.

Some scientists think that the precursors to full eusociality were insects that worked within a nest - more like a co-op - and not in an enormous colony.

This would apply to early ants, and maybe even to some species of hell ants.

Right now, though, it’s impossible to say whether true eusociality was the thing that distinguished the ancestors of modern ants from the hell ants.

But it does seem like the ant species that formed bigger colonies have been more successful.

So, where’d the hell ants go?

Well, there’s a 20-million-year gap in the fossil record between the last Cretaceous ant and the earliest Cenozoic ant fossils.

This huge gap in the fossil record sits right around the K-Pg extinction event 66 million years ago.

But after that time, ant abundance increased by an order of magnitude, for reasons that scientists don’t really understand.

It might have had something to do with the ants shifting from small colonies of a few hundred individuals to much, much bigger ones.

And while the overall abundance of modern ant ancestors increased, the diversity of ants decreased - basically, we lost the weird Mesozoic types and it’s not really clear why.

As for the hell ants, well they were gone about 10 million years before the Cenozoic even started.

And it might've had something to do with their wild jaws.

These predatory ants are thought to have used their specialized mandibles to capture and eat other insects...like that young cockroach.

And it's been shown that specialization in animals can increase their risk of becoming extinct.

So it’s possible that the weird mouthparts of hell ants made them more susceptible to extinction as flowering plants took over most ecosystems towards the end of the Cretaceous.

Or maybe it had to do with the fact that scientists don’t think they were living in huge colonies.

And even if they had some characteristics of eusocial insects, hell ants might not have been able to survive if they were only in groups of 100 or so.

Whatever the case, the extinction of the hell ants is still a puzzle for paleo entomologists.

But what they do know is that during the 20 million years that hell ants roamed the world